<html>
<head><meta charset="utf-8"><title>quinn · wg-async-foundations · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/index.html">wg-async-foundations</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/quinn.html">quinn</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="195976754"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/quinn/near/195976754" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/quinn.html#195976754">(May 01 2020 at 16:09)</a>:</h4>
<p>Hey <span class="user-mention" data-user-id="226095">@Dirkjan Ochtman</span> -- I have a question about <a href="https://github.com/djc/quinn" title="https://github.com/djc/quinn">quinn</a>. I am curious what it would take to make it portable across executors (or if it already is, even). From persuing the API, I see that when you connect with a stream, you get back something that implements <code>AsyncRead</code> (from the futures crate), but it seems like creating connections requires supplying addresses and things, and I <em>believe</em> that this sort of work is still something that's going to be tied to a specific async runtime. Is that all correct?</p>



<a name="195976771"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/quinn/near/195976771" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/quinn.html#195976771">(May 01 2020 at 16:09)</a>:</h4>
<p>(Others can feel free to chime in :)</p>



<a name="195976873"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/quinn/near/195976873" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/quinn.html#195976873">(May 01 2020 at 16:10)</a>:</h4>
<p>My mental model of where QUIC fits is probably a bit off anyway, I should go read more about it.</p>



<a name="195989541"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/quinn/near/195989541" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Benjamin Saunders <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/quinn.html#195989541">(May 01 2020 at 17:57)</a>:</h4>
<p>Hi, other quinn maintainer here! We rely on the async runtime (in quinn, not quinn-proto) to provide readiness notifications on a UDP socket and timers. There shouldn't be any serious barriers to abstracting those out.</p>



<a name="195989689"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/quinn/near/195989689" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Benjamin Saunders <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/quinn.html#195989689">(May 01 2020 at 17:58)</a>:</h4>
<p>I believe the parity folks are using quinn-proto, which has no async code per se, with async-std already</p>



<a name="195994189"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/quinn/near/195994189" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Benjamin Saunders <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/quinn.html#195994189">(May 01 2020 at 18:39)</a>:</h4>
<p>the code under <a href="https://github.com/djc/quinn/tree/master/quinn/src/platform" title="https://github.com/djc/quinn/tree/master/quinn/src/platform">https://github.com/djc/quinn/tree/master/quinn/src/platform</a> would need to be refactored to operate on a raw FD and some sort of abstract readiness notification interface, and that interface would need to be piped up to the <code>EndpointBuilder</code>.</p>



<a name="195998151"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/quinn/near/195998151" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/quinn.html#195998151">(May 01 2020 at 19:15)</a>:</h4>
<p>very interesting, thanks!</p>



<a name="195998190"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/quinn/near/195998190" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/quinn.html#195998190">(May 01 2020 at 19:15)</a>:</h4>
<p>I was just talking to <span class="user-mention" data-user-id="128448">@stjepang</span> about smol and in that conversation this idea of the "raw fd as a portability layer" kind of arose too...</p>



<a name="195998392"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/quinn/near/195998392" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Benjamin Saunders <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/quinn.html#195998392">(May 01 2020 at 19:17)</a>:</h4>
<p>np, feel free to ping me with any questions!<br>
note that managing readiness explicitly, rather than just calling a dynamically dispatched <code>send_to</code>, is important here because we need APIs from libc that aren't exposed by any higher level interface</p>



<a name="195998449"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/quinn/near/195998449" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Benjamin Saunders <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/quinn.html#195998449">(May 01 2020 at 19:18)</a>:</h4>
<p>and the degree of that use is only going to grow as we spend more time on optimization</p>



<a name="195999618"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/quinn/near/195999618" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dirkjan Ochtman <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/quinn.html#195999618">(May 01 2020 at 19:28)</a>:</h4>
<p>Also we currently spawn futures implicitly, so we'd need some abstraction around that</p>



<a name="195999673"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/quinn/near/195999673" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dirkjan Ochtman <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/quinn.html#195999673">(May 01 2020 at 19:29)</a>:</h4>
<p>but I don't think we have any particularly deep integration with tokio</p>



<a name="196000126"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/quinn/near/196000126" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dirkjan Ochtman <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/quinn.html#196000126">(May 01 2020 at 19:33)</a>:</h4>
<p>This is everything from tokio we use in quinn:</p>
<div class="codehilite"><pre><span></span><code><span class="k">use</span><span class="w"> </span><span class="n">tokio</span>::<span class="n">time</span>::<span class="p">{</span><span class="n">delay_until</span><span class="p">,</span><span class="w"> </span><span class="n">Delay</span><span class="p">,</span><span class="w"> </span><span class="n">Instant</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="n">TokioInstant</span><span class="p">};</span><span class="w"></span>
<span class="k">use</span><span class="w"> </span><span class="n">tokio</span>::<span class="n">io</span>::<span class="n">PollEvented</span><span class="p">;</span><span class="w"></span>
<span class="n">tokio</span>::<span class="n">spawn</span><span class="p">()</span><span class="w"></span>
<span class="n">AsyncRead</span><span class="w"> </span><span class="n">and</span><span class="w"> </span><span class="n">AsyncWrite</span><span class="w"> </span><span class="p">(</span><span class="n">but</span><span class="w"> </span><span class="n">we</span><span class="w"> </span><span class="n">also</span><span class="w"> </span><span class="k">impl</span><span class="w"> </span><span class="n">futures</span>::<span class="n">AsyncRead</span><span class="w"> </span><span class="n">and</span><span class="w"> </span><span class="n">futures</span>::<span class="n">AsyncWrite</span><span class="p">)</span><span class="w"></span>
</code></pre></div>



<a name="196000158"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/quinn/near/196000158" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dirkjan Ochtman <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/quinn.html#196000158">(May 01 2020 at 19:33)</a>:</h4>
<p>plus the <code>UdpSocket</code> directly from mio</p>



<a name="196003037"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/quinn/near/196003037" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dirkjan Ochtman <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/quinn.html#196003037">(May 01 2020 at 19:59)</a>:</h4>
<p>(never mind, added above)</p>



<a name="196011744"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/quinn/near/196011744" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> stjepang <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/quinn.html#196011744">(May 01 2020 at 21:23)</a>:</h4>
<p><span class="user-mention" data-user-id="116009">@nikomatsakis</span> just to add to what <span class="user-mention" data-user-id="226095">@Dirkjan Ochtman</span>  said - both timers and <code>PollEvented</code> (similar to my <code>Async</code> type) could be part of the universal reactor, if standard library is going to provide one</p>



<a name="196024979"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/quinn/near/196024979" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Matthias247 <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/quinn.html#196024979">(May 02 2020 at 00:52)</a>:</h4>
<p>As someone who has implemented all kinds of protocols (including complex ones like HTTP/2), and currently also looking at QUIC: You can model all of those based on a few interfaces which abstract away IO. What you typically need is what <span class="user-mention" data-user-id="226095">@Dirkjan Ochtman</span> already mentioned:</p>
<ul>
<li>IO (either in the form of stream based IO or packet based IO)</li>
<li>Timers</li>
<li>The ability to spawn subtasks</li>
</ul>
<p>I think all the libraries out there could have already decoupled themselves from tokio/async-std/etc, by simply building on top of those interfaces - which could be integrated in the libraries themselves. The reason why it doesn't happen: It introduces more work, is not obvious to do, and there is no real benefit for the library authors.</p>



<a name="196025120"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/quinn/near/196025120" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Matthias247 <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/quinn.html#196025120">(May 02 2020 at 00:54)</a>:</h4>
<blockquote>
<p>I was just talking to @stjepang about smol and in that conversation this idea of the "raw fd as a portability layer" kind of arose too...</p>
</blockquote>
<p>I think that is a rather bad portability layer. <code>fd</code>s are a unix concept. On windows they play a different role. Fuchsia and other capability based OSes have different assumptions again. And even on unix it doesn't work great, because different OSes and even OS subsystems have different assumptions on whether the fd can actually be async. Most things can not be.</p>



<a name="196025182"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/quinn/near/196025182" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Matthias247 <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/quinn.html#196025182">(May 02 2020 at 00:56)</a>:</h4>
<p>If you want an async stream abstraction or a timer abstraction, model it as such an abstraction. Then the platform can adept to whatever it offers.</p>



<a name="196027211"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/quinn/near/196027211" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Benjamin Saunders <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/quinn.html#196027211">(May 02 2020 at 01:54)</a>:</h4>
<p>to be clear, a generic packet-based I/O abstraction is not suitable here as we need a substantial amount of low-level platform specific shenanigans to achieve correct and efficient implementation</p>



<a name="196029074"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/quinn/near/196029074" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Matthias247 <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/quinn.html#196029074">(May 02 2020 at 02:48)</a>:</h4>
<p>The abstraction obviously needs to cover everything that the library needs. That might be a far bigger surface area than anyone who is not working on the library could imagine. That's why I think those abstractions are better off as part part of the libraries themselves.</p>
<p>For packet based projects, I've e.g. seen projects which support AF_XDP, DPDK, recvmmsg/sendmmsg and the regular socket interfaces. That's a huge surface area, and probably something that could be bikeshedded over for years if would need to get standardized. But if the library itself comes with a platform abstraction layer with all the features it requires people are still able to plug in different backends.</p>



<a name="196030491"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/quinn/near/196030491" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Benjamin Saunders <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/quinn.html#196030491">(May 02 2020 at 03:28)</a>:</h4>
<p>yeah, that's true; we in fact already have such a trait for abstracting over platforms' UDP stuff, just not exposed, and it's very simple</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>